GH-50994: [C++][Compute] Implement casting from ListView to List with zero-copy fast-path by Jay846 · Pull Request #50976 · apache/arrow

Jay846 · 2026-08-24T20:21:09Z

Rationale for this change

This PR implements missing type-casting compute kernels to convert ListViewType and LargeListViewType arrays to standard ListType and LargeListType arrays.

Previously, these casts routed to CastList, which ignored the sizes buffer and read offsets out-of-bounds, resulting in corrupted output arrays. This PR introduces a dedicated CastListView functor to perform correct conversions.

What changes are included in this PR?

To maximize performance and optimize memory layouts, a dual-execution path was implemented in the CastListView execution functor inside scalar_cast_nested.cc:

Contiguous Zero-Copy Fast-Path: Triggered when the input list-view elements are contiguous and adjacent (i.e. offsets[i] + sizes[i] == offsets[i+1]). It avoids copying the child values array entirely, allocating the new output offset buffer, shifting offsets relative to the start, and slicing the child array directly to preserve zero-copy pointer semantics.
Non-Contiguous Fallback Path: Triggered for gapped, overlapping, or out-of-order layouts. It dynamically tracks target offsets, builds integer index mappings using an Int64Builder, and invokes Arrow's internal take compute kernel to reconstruct a new contiguous child values array.
Kernel Registration: Formally wired into Arrow's casting system for standard and large variants of both nested types, replacing the incorrect routing.

Are these changes tested?

Yes, added comprehensive unit test suites in scalar_cast_test.cc passing all cases. Tests explicitly cover:

Contiguous Arrays: Validated zero-copy memory behavior programmatically by ensuring the source and destination child buffers share the exact same physical memory address.
Gapped & Overlapping Arrays: Verified correct offset tracking and reconstruction when offsets intersect or contain unused slots.
Null Propagation: Checked that null validity maps propagate correctly at both the parent and child levels.
Generic/Nested Types: Validated casting on non-primitive data types (e.g. utf8 strings) and recursive child type promotions (e.g., ListView<int16> to List<int32>).
Boundary Slices: Verified that taking slices of non-contiguous arrays computes correct offsets and avoids out-of-bound errors.

Are there any user-facing changes?

No public API contracts were broken. This adds correct, declarative casting support natively to the existing internal compute framework.

GitHub Issue: [C++][Compute] Implement casting from ListView/LargeListView to List/LargeList #50994

github-actions · 2026-08-24T20:21:33Z

Thanks for opening a pull request!

This pull request has been automatically converted to a draft because its title doesn't match Arrow's required format.

If this is not a minor PR. Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose

Opening GitHub issues ahead of time contributes to the Openness of the Apache Arrow project.

Then could you also rename the pull request title in the following format?

GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

or

MINOR: [${COMPONENT}] ${SUMMARY}

After updating the title, you can mark the pull request as ready for review.

See also:

Reranko05

Could you use the Arrow PR title template:-

GH-<Issue Number>: [<Component>] <Title>

github-actions · 2026-08-24T22:54:07Z

⚠️ GitHub issue #50976 has been automatically assigned in GitHub to PR creator.

HuaHuaY · 2026-08-25T02:48:37Z

You should use issue id but not the PR's id. If there isn't an existed issue, you can create a new one.

github-actions · 2026-08-25T16:08:58Z

⚠️ GitHub issue #50994 has been automatically assigned in GitHub to PR creator.

Jay846 · 2026-08-26T04:46:44Z

Hi @pitrou, I've updated the PR title to match the tracking issue (GH-50994), removed the unused compile variables, and applied clang-format styling. The failures in macOS GLib and Windows/Conda seem to be flaky Flight and S3FS/MinIO test failures. Could we please trigger a re-run of the checks? Thanks!

HuaHuaY · 2026-08-26T05:02:19Z

The failures in macOS GLib and Windows/Conda seem to be flaky Flight and S3FS/MinIO test failures.

Some CI tests are currently failing to run, but this is unrelated to the changes in this PR. Don't worry.

…ues, and Flatten

Jay846 · 2026-08-26T15:35:50Z

Also pushed a quick style formatting update to satisfy the pre-commit linter check. Kindly approve for re-run. Thanks

HuaHuaY · 2026-08-26T16:38:30Z

https://www.githubstatus.com/ shows that GitHub Actions is currently experiencing some issues. Let's wait for it to recover.

HuaHuaY

Left a few comments. I'm not sure about the check for in_array.length == 0. The rest looks good to me.

HuaHuaY · 2026-08-26T17:08:00Z

+    const ArraySpan& in_array = batch[0].array;
+    ArrayData* out_array = out->array_data().get();
+
+    if (in_array.length == 0) {


I may have a mistake during the previous review. I am not sure whether this if condition will always evaluate to false due to the check at cpp/src/arrow/compute/exec.cc:786; perhaps we can assume here that in_array.length is never 0. Let's wait for comments from a reviewer who is more familiar with Arrow Compute.

Got it, using if constexpr makes perfect sense here. I'll update those two checks.

For the length == 0 condition, I'll leave the check in place for now as a safeguard and wait for input from other maintainers on whether empty batches can reach this execution path. Thanks

feat(compute): implement casting from list_view to list

125ab1c

Jay846 requested a review from pitrou as a code owner August 24, 2026 20:21

github-actions Bot marked this pull request as draft August 24, 2026 20:21

github-actions Bot added Component: C++ awaiting review Awaiting review labels Aug 24, 2026

Reranko05 reviewed Aug 24, 2026

View reviewed changes

Jay846 changed the title ~~feat(compute): implement casting from list_view to list~~ [C++][Compute] Implement casting from ListView to List with zero-copy fast-path Aug 24, 2026

Jay846 changed the title ~~[C++][Compute] Implement casting from ListView to List with zero-copy fast-path~~ GH-50976: [C++][Compute] Implement casting from ListView to List with zero-copy fast-path Aug 24, 2026

github-actions Bot assigned Jay846 Aug 24, 2026

Jay846 marked this pull request as ready for review August 24, 2026 22:54

github-actions Bot added awaiting committer review Awaiting committer review and removed awaiting review Awaiting review labels Aug 25, 2026

style(compute): apply clang-format and clean up unused variables

289afef

Jay846 changed the title ~~GH-50976: [C++][Compute] Implement casting from ListView to List with zero-copy fast-path~~ GH-50994: [C++][Compute] Implement casting from ListView to List with zero-copy fast-path Aug 25, 2026

HuaHuaY self-requested a review August 26, 2026 05:04

HuaHuaY reviewed Aug 26, 2026

View reviewed changes

Comment thread cpp/src/arrow/compute/kernels/scalar_cast_nested.cc Outdated

Comment thread cpp/src/arrow/compute/kernels/scalar_cast_test.cc

fix(compute): remove unused iostream header in scalar_cast_nested

53085f0

HuaHuaY reviewed Aug 26, 2026

View reviewed changes

Comment thread cpp/src/arrow/compute/kernels/scalar_cast_nested.cc Outdated

Comment thread cpp/src/arrow/compute/kernels/scalar_cast_test.cc

fix(compute): correct downcast overflow checks in CastListView

beb87f7

HuaHuaY reviewed Aug 26, 2026

View reviewed changes

Comment thread cpp/src/arrow/compute/kernels/scalar_cast_nested.cc Outdated

Comment thread cpp/src/arrow/compute/kernels/scalar_cast_nested.cc

Comment thread cpp/src/arrow/compute/kernels/scalar_cast_nested.cc Outdated

Comment thread cpp/src/arrow/compute/kernels/scalar_cast_nested.cc Outdated

Jay Salvi added 2 commits August 26, 2026 18:31

fix(compute): address CastListView feedback on empty arrays, null val…

ce16dcc

…ues, and Flatten

style(compute): apply linter formatting suggestions to CastListView

68def9d

HuaHuaY reviewed Aug 26, 2026

View reviewed changes

refactor(compute): use if constexpr for is_downcast in CastListView

5e4cf34

Uh oh!

Conversation

Jay846 commented Aug 24, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

github-actions Bot commented Aug 24, 2026

Uh oh!

Reranko05 left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Aug 24, 2026

Uh oh!

HuaHuaY commented Aug 25, 2026

Uh oh!

github-actions Bot commented Aug 25, 2026

Uh oh!

Jay846 commented Aug 26, 2026

Uh oh!

HuaHuaY commented Aug 26, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Jay846 commented Aug 26, 2026

Uh oh!

HuaHuaY commented Aug 26, 2026

Uh oh!

HuaHuaY left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

HuaHuaY Aug 26, 2026

Choose a reason for hiding this comment

Uh oh!

Jay846 Aug 26, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Jay846 commented Aug 24, 2026 •

edited by github-actions Bot

Loading